Prediction device, prediction method, and storage medium

ABSTRACT

A prediction device including: a recognizer recognizing a road structure and another vehicle in the vicinity of a subject vehicle; and a predictor predicting a running locus of the other vehicle recognized by the recognizer in the future on the basis of the road structure recognized by the recognizer in a predetermined situation, wherein, in the predetermined situation, in a case in which at least a part of the road structure used for predicting the running locus of the other vehicle in the future is not recognizable for the recognizer, the predictor predicts the running locus of the other vehicle in the future on the basis of a running locus of the other vehicle in the past acquired on the basis of a result of recognition in the past that is acquired by the recognizer.

CROSS-REFERENCE TO RELATED APPLICATION

Priority is claimed on Japanese Patent Application No. 2018-007693,filed Jan. 19, 2018, the content of which is incorporated herein byreference.

BACKGROUND Field of the Invention

The present invention relates to a prediction device, a predictionmethod, and a storage medium.

Description of Related Art

Conventionally, predicting an approach between a subject vehicle andanother vehicle by predicting a traveling locus of the other vehicle forvarious uses has been researched. For example, in Japanese UnexaminedPatent Application First Publication No. 2011-209919, a technology hasbeen disclosed in which a predicted traveling route of a subject vehicleis specified on the basis of the current position of the subject vehicleand map data, positions of map-generated intersection points on thepredicted traveling route are set, a direction of the vehicle at eachpoint is predicted on the basis of a predicted locus passing througheach point, an attention area in which attention is required at the timeof traveling is specified, a range of directions in which the attentionarea is present as seen from a vehicle located at a map-generatedintersection point in the predicted direction is calculated, and anintersection map representing a correspondence relationship between adirection as seen from the vehicle and a degree of risk is generated onthe basis of the calculated range of directions.

SUMMARY

In this conventional technology, it is necessary to be able to recognizea road structure (road shape) as a premise of the prediction, andaccordingly, applicable situations are limited. In order to recognize aroad structure, structures partitioning a road, for example, acurbstone, a road partition line, and the like need to be recognizedusing a camera, a laser radar, or the like, for example, but in asituation in which an oncoming vehicle turns right or left and traversesin front of the subject vehicle at an intersection, structures may entera blind area due to the oncoming vehicle and may not be able to berecognized. For this reason, in the conventional technology, there arecases in which a traveling locus of another vehicle in the future maynot be able to be continuously predicted.

An aspect of the present invention is in consideration of suchsituations, and one object thereof is to provide a prediction device, aprediction method, and a storage medium capable of continuouslypredicting running loci of other vehicles in the future.

A prediction device, a prediction method, and a storage medium accordingto the present invention employ the following configurations.

(1): According to one aspect of the present invention, there is provideda prediction device including: a recognizer recognizing a road structureand another vehicle in the vicinity of a subject vehicle; and apredictor predicting a running locus of the other vehicle recognized bythe recognizer in the future on the basis of the road structurerecognized by the recognizer in a predetermined situation, wherein, inthe predetermined situation, in a case in which at least a part of theroad structure used for predicting the running locus of the othervehicle in the future is not recognizable for the recognizer, thepredictor predicts the running locus of the other vehicle in the futureon the basis of a running locus of the other vehicle in the pastacquired on the basis of a result of recognition in the past that hasbeen acquired by the recognizer.

(2): In the aspect (1) described above, the predetermined situation is asituation in which the other vehicle changes a course at anintersection.

(3): In the aspect (2) described above, the predictor predicts therunning locus of the other vehicle in the future on the basis ofpositions of an entrance and an exit of the intersection through whichthe other vehicle passes, which are acquired from the road structurerecognized by the recognizer, and predicts the running locus of theother vehicle in the future on the basis of the running locus of theother vehicle in the past in a case in which one or both of thepositions of the entrance and the exit are not clear.

(4): In the aspect (3) described above, the predictor predicts therunning locus of the other vehicle in the future by correcting aprovisional running locus predicted from the running locus of the othervehicle in the past on the basis of the position of the exit in a casein which the position of the exit is recognized by the recognizer, andthe position of the entrance is not recognizable for the recognizer.

(5): In the aspect (3) described above, in a case in which the positionof the exit is not recognizable for the recognizer, the predictorestimates the position of the exit on the basis of a running locus of apreceding vehicle in the past running in front of the other vehicle thatis a target for predicting the running locus in the future and predictsthe running locus of the other vehicle in the future.

(6): In the aspect (3) described above, in a case in which the positionof the exit is not recognizable for the recognizer, the predictorestimates the position of the exit on the basis of the road structure ona side facing the exit at the intersection and predicts the runninglocus of the other vehicle in the future on the basis of the estimatedposition of the exit.

(7): In the aspect (3) described above, in a case in which the positionof the exit is not recognizable for the recognizer, the predictorestimates the position of the exit on the basis of a road width on aside facing the exit at the intersection and predicts the running locusof the other vehicle in the future on the basis of the estimatedposition of the exit.

(8): In the aspect (3) described above, in a case in which the positionof the exit is not recognizable for the recognizer, the predictorestimates the position of the exit on the basis of a position of astructure part disposed near the exit and predicts the running locus ofthe other vehicle in the future on the basis of the estimated positionof the exit.

(9): In the aspect (1) described above, the predetermined situation is asituation in which the other vehicle crosses a road and advances out ofthe road in a case in which the other vehicle is an oncoming vehicle.

(10): A prediction method according to another aspect of the presentinvention is a prediction method using a computer including: recognizinga road structure and another vehicle in the vicinity of a subjectvehicle; predicting a running locus of the other vehicle in the futureon the basis of the recognized road structure in a predeterminedsituation; and predicting the running locus of the other vehicle in thefuture on the basis of a running locus of the other vehicle in the pastacquired on the basis of a result of recognition in the past in a casein which at least a part of the road structure used for predicting therunning locus of the other vehicle in the future is not recognizable inthe predetermined situation.

(11): A storage medium according to another aspect of the presentinvention is a computer-readable non-transitory storage medium storing aprogram thereon, the program causing a computer to execute: recognizinga road structure and another vehicle in the vicinity of a subjectvehicle; predicting a running locus of the other vehicle in the futureon the basis of the recognized road structure in a predeterminedsituation; and predicting the running locus of the other vehicle in thefuture on the basis of a running locus of the other vehicle in the pastacquired on the basis of a result of recognition in the past in a casein which at least a part of the road structure used for predicting therunning locus of the other vehicle in the future is not recognizable inthe predetermined situation.

According to (1) to (11), running loci of other vehicles in the futurecan be predicted continuously.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a configuration diagram of a vehicle system using a predictiondevice;

FIG. 2 is a functional configuration diagram of a first controller and asecond controller;

FIG. 3 is a diagram illustrating one example of “a predeterminedsituation” in which another vehicle m1 that is a prediction targetpasses through an intersection;

FIG. 4 is a diagram illustrating a process executed by a running locuspredictor in a case in which an entrance and an exit of an intersectionare recognized;

FIG. 5 is a diagram illustrating two patterns of circular arcs that areset;

FIG. 6 is a diagram (1) illustrating a process executed by a runninglocus predictor in a case in which an entrance of an intersection cannotbe recognized;

FIG. 7 is a diagram (2) illustrating a process executed by the runninglocus predictor in a case in which an entrance of an intersection cannotbe recognized;

FIG. 8 is a diagram (1) illustrating a process executed by the runninglocus predictor in a case in which an exit of an intersection cannot berecognized;

FIG. 9 is a diagram (2) illustrating a process executed by the runninglocus predictor in a case in which an exit of an intersection cannot berecognized;

FIG. 10 is a flowchart illustrating one example of the flow of a processexecuted by the running locus predictor;

FIG. 11 is a diagram illustrating one example of a “predeterminedsituation” in which another vehicle that is an oncoming vehicle passesthrough an intersection;

FIG. 12 is a diagram illustrating a process executed by the runninglocus predictor;

FIG. 13 is a diagram illustrating a process executed by the runninglocus predictor in a case in which at least a part of an opposing lanecannot be recognized; and

FIG. 14 is a diagram illustrating one example of the hardwareconfiguration of an automatic driving control device according to anembodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a prediction device, a prediction method, and a storagemedium according to embodiments of the present invention will bedescribed with reference to the drawings. A prediction device is usedfor various purposes. In the following description, although theprediction device will be described as being applied to an automaticdriving vehicle, the prediction device can be applied also to an alarmdevice that outputs an alarm in accordance with an approach of anothervehicle, a driving supporting device that performs inter-vehicledistance control, lane keep supporting control, and the like andperforms switching of control or adjusting the amount of control inaccordance with a locus of another vehicle in the future, and the like.

[Configuration]

FIG. 1 is a configuration diagram of a vehicle system 1 using aprediction device. A vehicle in which the vehicle system 1 is mountedis, for example, a vehicle having two wheels, three wheels, four wheels,or the like, and a driving source thereof is an internal combustionengine such as a diesel engine or a gasoline engine, an electric motor,or a combination thereof. In a case in which an electric motor isincluded, the electric motor operates using electric power generatedusing a power generator connected to an internal combustion engine ordischarge power of a secondary cell or a fuel cell.

The vehicle system 1, for example, includes a camera 10, a radar device12, a finder 14, an object recognizing device 16, a communication device20, a human machine interface (HMI) 30, a vehicle sensor 40, anavigation device 50, a map positioning unit (MPU) 60, a drivingoperator 80, an automatic driving control device 100, a running drivingforce output device 200, a brake device 210, and a steering device 220.Such devices and units are interconnected using a multiplexcommunication line such as a controller area network (CAN) communicationline, a serial communication line, a radio communication network, or thelike. The configuration illustrated in FIG. 1 is merely one example, andthus, a part of the configuration may be omitted, and, furthermore,other components may be added thereto.

The camera 10, for example, is a digital camera using a solid-stateimaging device such as a charge coupled device (CCD) or a complementarymetal oxide semiconductor (CMOS). One or a plurality of cameras 10 areinstalled at arbitrary places on a vehicle (hereinafter, referred to asa subject vehicle M) in which the vehicle system 1 is mounted. In a casein which the side in front is to be imaged, the camera 10 is installedat an upper part of a front windshield, a rear face of a rear-viewmirror, or the like. The camera 10, for example, repeatedly images thevicinity of the subject vehicle M periodically. The camera 10 may be astereo camera.

The radar device 12 emits radiowaves such as millimeter waves to thevicinity of the subject vehicle M and detects at least a position of (adistance and an azimuth to) an object by detecting radiowaves (reflectedwaves) reflected by the object. One or a plurality of radar devices 12are installed at arbitrary places on the subject vehicle M. The radardevice 12 may detect a position and a speed of an object using afrequency modulated continuous wave (FM-CW) system.

The finder 14 is a light detection and ranging (LIDAR) device. Thefinder 14 emits light to the vicinity of the subject vehicle M andmeasures scattering light. The finder 14 detects a distance to a targeton the basis of a time from light emission to light reception. Theemitted light, for example, is a pulse-form laser light. One or aplurality of finders 14 are mounted at arbitrary positions on thesubject vehicle M. The finder 14 is one example of an object detectingdevice.

The object recognizing device 16 may perform a sensor fusion process onresults of detection using some or all of the camera 10, the radardevice 12, and the finder 14, thereby allowing recognition of aposition, a type, a speed, and the like of an object. The objectrecognizing device 16 outputs a result of recognition to the automaticdriving control device 100. The object recognizing device 16, as isnecessary, may output results of detection using the camera 10, theradar device 12, and the finder 14 to the automatic driving controldevice 100 as they are.

The communication device 20, for example, communicates with othervehicles present in the vicinity of the subject vehicle M using acellular network, a Wi-Fi network, BLUETOOTH®, dedicated short rangecommunication (DSRC), or the like or communicates with various serverapparatuses through a radio base station.

The HMI 30 presents various types of information to an occupant of thesubject vehicle M and receives an input operation performed by a vehicleoccupant. The HMI 30 may include various display devices, a speaker, abuzzer, a touch panel, switches, keys, and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects aspeed of the subject vehicle M, an acceleration sensor that detects anacceleration, a yaw rate sensor that detects an angular velocity arounda vertical axis, an azimuth sensor that detects the azimuth of thesubject vehicle M, and the like.

The navigation device 50, for example, includes a global navigationsatellite system (GNSS) receiver 51, a navigation HMI 52, and a routedeterminer 53 and stores first map information 54 in a storage devicesuch as a hard disk drive (HDD) or a flash memory. The GNSS receiver 51identifies a position of a subject vehicle M on the basis of signalsreceived from GNSS satellites. The position of the subject vehicle M maybe identified or complemented by an inertial navigation system (INS)using an output of the vehicle sensor 40. The navigation HMI 52 includesa display device, a speaker, a touch panel, a key, and the like. A partor the whole of the navigation HMI 52 and the HMI 30 described above maybe configured to be shared. The route determiner 53, for example,determines a route to a destination input by a vehicle occupant usingthe navigation HMI 52 (hereinafter, referred to as a route on a map)from a position of the subject vehicle M identified by the GNSS receiver51 (or an input arbitrary position) by referring to the first mapinformation 54. The first map information 54, for example, isinformation in which a road form is represented by respective linksrepresenting a road and respective nodes connected using the links. Theroute on the map determined by the route determiner 53 is output to theMPU 60. The navigation device 50 may perform route guidance using thenavigation HMI 52 on the basis of the route on the map determined by theroute determiner 53. The navigation device 50, for example, may berealized by a function of a terminal device such as a smartphone or atablet terminal held by a vehicle occupant. The navigation device 50 maytransmit a current location and a destination to a navigation serverthrough the communication device 20 and acquire a route on the mapreceived from the navigation server as a reply.

The MPU 60, for example, functions as a recommended lane determiner 61and stores second map information 62 in a storage device such as an HDDor a flash memory. The recommended lane determiner 61 divides a routeprovided from the navigation device 50 into a plurality of blocks (forexample, divides the route into blocks of 100 [m] in the advancementdirection of the vehicle) and determines a recommended lane for eachblock by referring to the second map information 62. In a case in whicha branching place, a merging place, or the like is present in the route,the recommended lane determiner 61 determines a recommended lane suchthat the subject vehicle M can run on a reasonable route for advancementto divergent destinations.

The second map information 62 is map information having an accuracyhigher than that of the first map information 54. The second mapinformation 62, for example, includes information of the center of eachlane, information of a boundary between lanes, or the like. Roadinformation, traffic regulation information, address information (anaddress and a postal code), facility information, telephone numberinformation, and the like may be included in the second map information62. The second map information 62 may be updated as is necessary byaccessing another device using the communication device 20.

The driving operator 80, for example, includes an acceleration pedal, abrake pedal, a shift lever, a steering wheel, a steering wheel variant,a joystick, and other operators. A sensor detecting the amount of anoperation or the presence/absence of an operation is installed in thedriving operator 80, and a result of the detection is output to theautomatic driving control device 100 or at least one or all of therunning driving force output device 200, the brake device 210, and thesteering device 220.

The automatic driving control device 100, for example, includes a firstcontroller 120 and a second controller 160. Each of the first controller120 and second controller 160, for example, is realized by a hardwareprocessor such as a central processing unit (CPU) executing a program(software). Some or all of such constituent elements may be realized byhardware (a circuit unit; including circuitry) such as a large scaleintegration (LSI), an application specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or a graphics processing unit(GPU) or may be realized by cooperation between software and hardware.The program may be stored in a storage device (a storage deviceincluding a non-transitory storage medium) such as a hard disk drive(HDD) or a flash memory in advance or may be stored in a storage medium(non-transitory storage medium) such as a DVD or a CD-ROM that can beloaded or unloaded and installed by loading the storage medium into adrive device. The automatic driving control device 100 is one example ofa vehicle control device.

FIG. 2 is a functional configuration diagram of the first controller 120and the second controller 160. The first controller 120, for example,includes a recognizer 130, a running locus predictor 140, and an actionplan generator 150.

The recognizer 130 recognizes a surrounding status of the subjectvehicle M on the basis of information input from the camera 10, theradar device 12, and the finder 14 through the object recognizing device16. The recognizer 130, for example, includes a road structurerecognizer 132 and an other-vehicle recognizer 134. Various objects areincluded in a surrounding status recognized by the recognizer. Theposition of an object, for example, is recognized, first, as a positionon an absolute coordinate system having a representative point (thesensor position or the center of gravity, the center of a driving shaft,or the like) of the subject vehicle M as its origin, is converted into aposition on road coordinates along the road as is necessary, and is usedfor control.

The road structure recognizer 132 recognizes a road structure in thevicinity of the subject vehicle. For example, the road structurerecognizer 132 recognizes a road partition line, a curbstone, a medianstrip, a guardrail, and the like and recognizes a shape of a road (roadstructure) partitioned thereby. In the recognized road structure,various kinds of information such as a width of each lane, a position ofa connection part connected with an intersection. Details thereof willbe described later.

The other-vehicle recognizer 134 recognizes states of other vehiclessuch as positions, speeds, and accelerations that are present in thevicinity of the subject vehicle M. The positions of other vehicles maybe represented using representative points such as the centers ofgravity or corners of the other vehicles or may be represented usingareas having a spread.

The surrounding status recognized by the recognizer 130 may include arelative position and posture between the subject vehicle M and therunning lane, states of bicycles and pedestrians, road events such as atemporary stop line, an obstacle, a red traffic signal, and a tollgateand other information in addition to the road structure and othervehicles. A result of the recognition acquired by the recognizer 130 isoutput to the running locus predictor 140 and the action plan generator150.

The running locus predictor 140, in a predetermined situation, predictsrunning loci of the other vehicles, which have been recognized by theother-vehicle recognizer 134, in the future on the basis of the roadstructure recognized by the road structure recognizer 132. In apredetermined scene, in a case in which at least a part of the roadstructure used for predicting the running loci of the other vehicles inthe future cannot be recognized by the road structure recognizer 132 ina predetermined situation, the running locus predictor 140 predicts therunning loci of the other vehicles in the future on the basis of runningloci of vehicles in the past acquired on the basis of results ofrecognition acquired by the other-vehicle recognizer 134 in the past.Details thereof will be described later.

The action plan generator 150 generates a target locus along which thesubject vehicle M runs in the future such that it basically runs in arecommended lane determined by the recommended lane determiner 61 anddoes not interfere with the running loci of the other vehicles in thefuture predicted by the running locus predictor 140. The target locus,for example, includes a plurality of locus points and speed elements.For example, the target locus is represented by sequentially aligningplaces (locus points) at which the subject vehicle M is to arrive. Alocus point is a position at which the subject vehicle M is to arrivefor every predetermined running distance (for example about several [m])in terms of a distance along the road, and, separately from that, atarget speed and a target acceleration for every predetermined samplingtime (for example, a fraction of a second) are generated as a part ofthe target locus. A locus point may be a position at which the subjectvehicle M is to arrive at a sampling time among predetermined samplingtime. In such a case, information of the target speed and the targetacceleration is represented at the interval of locus points.

A combination of the recognizer 130 and the running locus predictor 140forms one example of a “prediction device”. A part of the function ofthe recognizer 130 described above may be included in the objectrecognizing device 16. In such a case, the “prediction device” mayinclude the object recognizing device 16. The “prediction device” mayinclude devices used for recognizing a road structure and othervehicles, for example, the camera 10, the radar device 12, and thefinder 14.

The second controller 160 performs control of the running driving forceoutput device 200, the brake device 210, and the steering device 220such that the subject vehicle M passes along a target locus generated bythe action plan generator 150 at a scheduled time. For example, thesecond controller 160 acquires information of a target locus (locuspoints) generated by the action plan generator 150, stores the acquiredinformation in a memory (not illustrated in the drawing), and controlsthe running driving force output device 200 or the brake device 210 onthe basis of a speed element accompanying the target locus stored in thememory. The second controller 160 controls the steering device 220 inaccordance with a bending state of the target locus stored in thememory.

The running driving force output device 200 outputs a running drivingforce (torque) used for a vehicle to run to driving wheels. The runningdriving force output device 200, for example, includes a combination ofan internal combustion engine, an electric motor, a transmission, andthe like and an ECU controlling these components. The ECU controls thecomponents described above in accordance with information input from thesecond controller 160 or information input from the driving operator 80.

The brake device 210, for example, includes a brake caliper, a cylinderthat delivers hydraulic pressure to the brake caliper, an electric motorthat generates hydraulic pressure in the cylinder, and a brake ECU. Thebrake ECU performs control of the electric motor in accordance withinformation input from the second controller 160 or information inputfrom the driving operator 80 such that a brake torque according to abrake operation is output to each vehicle wheel. The brake device 210may include a mechanism delivering hydraulic pressure generated inaccordance with an operation on the brake pedal included in the drivingoperators 80 to the cylinder through a master cylinder as a backup. Inaddition, the brake device 210 is not limited to the configurationdescribed above and may be an electronically-controlled hydraulic brakedevice that delivers hydraulic pressure in the master cylinder to acylinder by controlling an actuator in accordance with information inputfrom the second controller 160.

The steering device 220, for example, includes a steering ECU and anelectric motor. The electric motor, for example, changes the directionof the steering wheel by applying a force to a rack and pinionmechanism. The steering ECU changes the direction of the steering wheelby driving an electric motor in accordance with information input fromthe second controller 160 or information input from the driving operator80.

[Running Locus Prediction—Intersection]

Hereinafter, a prediction of running loci of other vehicles that isexecuted by cooperation between the recognizer 130 and the running locuspredictor 140 will be described. First, a process will be describedwhich is performed in a case in which the predetermined situation is asituation in which another vehicle changes a course at an intersection.FIG. 3 is a diagram illustrating one example of “a predeterminedsituation” in which another vehicle m1 that is a prediction targetpasses through an intersection. In FIG. 3, the subject vehicle M islocated at a position at which each road connected to an intersectioncan be looked over, and another vehicle m1 is entering into theintersection from a lane L1. In FIG. 3 and subsequent drawings, a sceneon an actual plane seen from the sky is illustrated. In description ofFIG. 3 and subsequent drawings, use in a left-traffic country or regionis premised. In the description of FIG. 3 and the subsequent drawings,an arrow attached to a vehicle represents an advancement direction ofthe vehicle.

In such a situation, the road structure recognizer 132 recognizes theposition of a place at which another vehicle m1 enters the intersection,in other words, an entrance CI of the intersection. The position of theentrance CI, for example, is defined as an intersection between a centerline of a lane L1 and a white line closest to a center part of theintersection (details thereof will be described with reference to FIG.4). The road structure recognizer 132 recognizes a position of a placeat which another vehicle m1 leaves from the intersection, in otherwords, an exit CO of the intersection. The exit CO is not limited to benarrowed down to one, and there are cases in which a plurality of exitsCO are recognized, and the exits are narrowed down to one exit through aprocess to be described later. In FIG. 3, lanes from which there is alikelihood that another vehicle m1 leaves are lanes L2 to L9. In thiscase, for example, the position of the exit CO is defined as anintersection between a center line of each lane and a white line closestto the center part of the intersection.

(Case in which Entrance and Exit are Recognized)

FIG. 4 is a diagram illustrating a process executed by the running locuspredictor 140 in a case in which an entrance CI and an exit CO of anintersection are recognized. The running locus predictor 140, first,virtually sets a center line CL1 of a lane L1 in which another vehiclem1 is running. Similarly, the running locus predictor 140 virtually setscenter lines CL2 to CL9 of lanes L2 to L9.

The running locus predictor 140, for example, extracts a center lineintersecting with a center line CL1 from center lines of lanes and setsa circular arc that is inscribed in both a center line CLx of a lane xhaving the extracted center line and the center line CL1 and passesthrough one of the entrance and the exit of the intersection. FIG. 5 isa diagram illustrating two patterns of circular arcs that are set. Acircular arc passing through the entrance CI of the intersection is setin a left diagram of FIG. 5, and a circular arc passing through the exitCO of the intersection is set in a right diagram of FIG. 5.

The running locus predictor 140 predicts a line in which the center lineCL1, the set circular arc described above, and the center line CLx arealigned as a running locus of another vehicle m1. For example, in thecase of the left diagram of FIG. 5, a line in which the center line CL1or the center line CLx is connected to both ends of a circular arcbetween the entrance CI and a contact point CP as a running locus ofanother vehicle m1 in the case of advancement to a lane x.

The running locus is acquired for each of lanes from which there is alikelihood that another vehicle m1 leaves. The running locus predictor140, for example, predicts one running locus in which the positions ofanother vehicle m1 overlap each other when seen from the sky among aplurality of running loci as a running locus of another vehicle.

(Case in which Entrance Cannot be Recognized)

FIG. 6 is a diagram (1) illustrating a process executed by the runninglocus predictor 140 in a case in which an entrance CI of an intersectioncannot be recognized. In the drawing, the entrance CI of theintersection is blocked by another vehicle m2 and becomes a blind areafor the camera 10 and the like of the subject vehicle M and thus, cannotbe recognized by the subject vehicle M. In this case, the running locuspredictor 140 estimates a running locus of another vehicle m1 in thefuture on the basis of the running locus of another vehicle m1 in thepast. For example, the running locus predictor 140 predicts a runninglocus in the future by applying a Kalman filter to a running locus ofanother vehicle m1 in the past that has been sampled for everypredetermined time. In the drawing, K represents sampling points of arunning locus in the past and in the future.

The principle of the Kalman filter will be briefly described. In a casein which a Kalman filter process is executed, the running locuspredictor 140, first, applies moving of another vehicle m1 to each of aplurality of models. As the models, for example, (1) running straightwhile maintaining the speed, (2) running straight while accelerating ordecelerating, (3) turning while maintaining the speed, (4) turning whileaccelerating or decelerating, and the like are prepared in advance.

The running locus predictor 140 selects a most applicable model. AKalman gain is realized by alternately performing a prediction steprepresented in Equations (1) and (2) and an update step represented inEquations (3) to (5). In each equation, k represents a process cycle.μ_(k) (tilde) represents an in-advance estimated value in Equation (1),and Σ_(k) (tilde) represents an in-advance error covariance in Equation(2). In addition, G_(k) is a matrix configuring a state spacerepresentation (operation model), and R is Σμ.{tilde over (μ)}_(k) g(μ_(k−1))  (1){tilde over (μ)}_(k) G _(k)Σ_(k−1) G _(k) ^(T) +R  (2)

In Equation (3), K_(k) represents a Kalman gain, C represents a matrixconfiguring a state space representation (observation model), and Qrepresents Σ_(z) (here, z is a left term of the observation model).μ_(k) represents a post estimated value in Equation (4), and Σ_(k)represents a post error covariance in Equation (5). μ_(k) represented inEquation (4) is handled as μ_(k+1) (tilde) at the (k+1)-th cycle, andΣ_(k) represented in Equation (5) is handled as Σ_(k+1) at the (k+1)-thcycle.K _(k){tilde over (Σ)}_(k) C ^(T)(C{tilde over (Σ)} _(k) C ^(T)+Q)⁻¹  (3)μ_(k)={tilde over (μ)}_(k) +K _(k)(z _(k) −C{tilde over (μ)} _(k))  (4)Σ_(k)=(I−K _(k) C){tilde over (Σ)}_(k)  (5)

In a case in which the running locus matches one of model (1) runningstraight while maintaining the speed and (2) running straight whiledecelerating/accelerating, the running locus predictor 140, asillustrated in FIG. 6, sets a straight line LK regarded as a runninglocus. In this case, in a case in which another vehicle m1 is predictedto make a left or right turn (not running straight) in accordance withan operation state of a direction indicator or deceleration of anothervehicle m1 or left/right turn information acquired by communicating withanother vehicle m1, the running locus predictor 140, for example, sets acircular arc that is inscribed in both the center line CLx of a focusedlane x and the straight line LK and passes through one of the currentposition of another vehicle m1 and the exit CO. Then, the running locuspredictor 140 predicts a line in which the straight line LK, the setcircular arc described above, and the center line CLx are aligned as arunning locus of another vehicle m1. In this way, the running locus(provisional running locus) predicted from the running locus of anothervehicle m1 in the past using the Kalman filter is corrected on the basisof the position of the exit CO, and a running locus of another vehiclem1 in the future is predicted. While there are cases in which runningloci are not narrowed down to one through a relating process, therunning locus predictor 140, for example, may switch the model to (3)turning while maintaining the speed or (4) turning while accelerating ordecelerating at a time point at which another vehicle m1 starts to turnand execute the following process described with reference to FIG. 7.

In a case in which the running locus matches one of the models (3)turning while maintaining the speed and (4) turning while acceleratingor decelerating, the running locus predictor 140, for example, asillustrated in FIG. 7, may set a straight line CK acquired by extendingthe model to the exit CO of the intersection, select a lane having asmallest correction reference deviation illustrated in the drawing, andcorrect the model of the Kalman filter such that the correctionreference deviation is negated. FIG. 7 is a diagram (2) illustrating aprocess executed by the running locus predictor 140 in a case in whichan entrance CI of an intersection cannot be recognized.

(Case in which Exit Cannot be Recognized)

FIG. 8 is a diagram (1) illustrating a process executed by the runninglocus predictor 140 in a case in which an exit CO of an intersectioncannot be recognized. In the drawing, exits leaving from theintersection toward lanes L7 and L9 are blocked by other vehicles m2 andm3 running in front of another vehicle m1 and cannot be recognized froma subject vehicle M. In this case, the running locus predictor 140, forexample, estimates that there is an exit CO toward the lane L7 at anadvancement destination of another vehicle m2 and estimates that thereis an exit CO toward the lane L9 at an advancement destination ofanother vehicle m3. An advancement destination, for example, is definedas an intersection between a running locus of another vehicle and awhite line RL closest to the center part of the intersection. Therunning locus predictor 140 performs a process similar to that of (Casein which entrance and exit are recognized) or (Case in which entrancecannot be recognized) described above on the basis of the position ofthe estimated exit CO.

FIG. 9 is a diagram (2) illustrating a process executed by the runninglocus predictor 140 in a case in which an exit CO of an intersectioncannot be recognized. In this drawing, a part of the intersectionillustrated in FIG. 3 and the like is illustrated in an enlarged scale.The running locus predictor 140, for example, sets virtual lines VL byvirtually extending road partition lines by referring to a roadstructure of a position facing lanes L7 to L9 leaving from theintersection. Then, the running locus predictor 140 sets a middle pointbetween two intersections MP between the virtual lines VL and anexit-side white line RL as the exit CO of the intersection.

The running locus predictor 140 may estimate the position of the exit COof the intersection on the basis of structures, for example, a curbstoneCS, a mark, a traffic light, a guardrail, and the like in a range seenfrom the subject vehicle M. In such a case, the running locus predictor140 may estimate the position of the exit CO of the intersection moreaccurately on the basis of a road width WR and/or a lane width WLrecognized from the road structure of the position facing the lanes L7to L9 leaving from the intersection.

[Process Flow]

FIG. 10 is a flowchart illustrating one example of the flow of a processexecuted by the running locus predictor 140. The process of thisflowchart, for example, is repeatedly executed.

First, the running locus predictor 140 determines whether or not thesubject vehicle M has approached the intersection (Step S100). Here,“approaching”, for example, means that the subject vehicle becomes lessthan a predetermined distance or a predetermined time to the entrance CIor the center point of the intersection. In a case in which the subjectvehicle M has not approached the intersection, one routine of thisflowchart ends.

On the other hand, in a case in which the subject vehicle M hasapproached the intersection, the running locus predictor 140 determineswhether or not there is another vehicle (hereinafter target vehicle) ofwhich a running locus to be predicted by referring to a result ofrecognition acquired by the other-vehicle recognizer 134 (Step S102). Atarget vehicle, for example, is a vehicle acquired by excluding othervehicles advancing along running loci not clearly intersecting with thesubject vehicle M among other vehicles present within the intersectionor approaching the intersection. In a case in which there is no targetvehicle, one routing of this flowchart ends.

On the other hand, in a case in which there is a target vehicle, therunning locus predictor 140 determines whether or not the road structurerecognizer 132 can recognize (or can recognized within a predeterminedtime; hereinafter, the same) both an entrance and an exit of theintersection (Step S104). In a case in which the road structurerecognizer 132 can recognize both an entrance and an exit of theintersection, the running locus predictor 140 predicts a running locusof the target vehicle on the basis of the position of the entrance andthe position of the exit of the intersection (further on the basis ofthe position of the target vehicle in the example described above) (StepS106).

On the other hand, in a case in which the road structure recognizer 132cannot recognize both an entrance and an exit of the intersection, therunning locus predictor 140 determines whether or not the road structurerecognizer 132 cannot recognize the entrance of the intersection and canrecognize the exit (Step S108). In a case in which the road structurerecognizer 132 cannot recognize the entrance of the intersection and canrecognize the exit, the running locus predictor 140 predicts a runninglocus of the target vehicle on the basis of a running history of thetarget vehicle in the past and the position of the exit of theintersection (Step S110).

In a case in which No is determined in Step S108, the running locuspredictor 140 determines whether or not the road structure recognizer132 cannot recognize the exit of the intersection and can recognize theentrance (Step S112). In a case in which the road structure recognizer132 cannot recognize the exit of the intersection and can recognize theentrance, the running locus predictor 140 estimates the position of theexit of the intersection on the basis of a running locus of a precedingvehicle of the target vehicle, a facing road structure, a curbstone, andthe like and predicts a running locus of the target vehicle on the basisof the position of the entrance of the intersection and the estimatedposition of the exit (Step S114).

On the other hand, in a case in which No is determined in Step S112 (ina case in which neither an entrance nor an exit of the intersectioncannot be recognized), the running locus predictor 140 predicts arunning locus of the target vehicle on the basis of the running historyof the target vehicle in the past and the position of the exit of theintersection estimated similar to Step S114 (Step S116).

In accordance a relating process, even in a case in which at least apart of the intersection cannot be recognized by the road structurerecognizer 132, a running locus of the target vehicle can beappropriately predicted.

[Running Locus Prediction—Road Crossing]

Hereinafter, a process performed in a case in which a predeterminedsituation is a situation in which another vehicle that is an oncomingvehicle is crossing a road will be described. FIG. 11 is a diagramillustrating one example of a “predetermined situation” in which anothervehicle m1 that is an oncoming vehicle passes through an intersection.In this situation, another vehicle m1 that is an oncoming vehicle forthe subject vehicle M is trying to cross a road and advances to aparking space P disposed outside the road.

FIG. 12 is a diagram illustrating a process executed by the runninglocus predictor 140. In this case, the running locus predictor 140, forexample, sets a center line CL of a lane in which another vehicle m1 isrunning and a virtual line EL acquired by extending from a center partCE of the entrance/exit of the parking space P in a road width directionand sets a circular arc that is inscribed in both the center line CL andthe virtual line EL and passes through one of the current position ofthe other vehicle m1 and the center part CE. Then, the running locuspredictor 140 predicts a line in which the center line CL, the circulararc set as above, and the virtual line EL are aligned as a running locusof another vehicle m1.

FIG. 13 is a diagram illustrating a process executed by the runninglocus predictor 140 in a case in which at least a part of an opposinglane cannot be recognized. In the example illustrated in the drawing, inaccordance with the presence of another vehicle m4, a part of a lane inwhich another vehicle m1 is running is blocked, and the center line CLillustrated in FIG. 12 cannot be correctly recognized. In this case,similar to the case of the intersection (a case in which an entrancecannot be recognized), the running locus predictor 140 estimates arunning locus of another vehicle m1 in the future on the basis of therunning locus of another vehicle m1 in the past. For example, therunning locus predictor 140 predicts a running locus of another vehiclem1 in the future by applying a Kalman filter to the running locus ofanother vehicle m1 in the past, which has been sampled for everypredetermined time, and corrects the running locus of another vehicle m1in the future using the position of the center part CE of theentrance/exit of the parking space P.

According to the prediction device, the prediction method, and thestorage medium according to the embodiment described above, therecognizer 130 that recognizes a road structure and other vehicles inthe vicinity of a subject vehicle and the running locus predictor 140that predicts a running locus of another vehicle in the future, whichhas been recognized by the recognizer 130, on the basis of the roadstructure recognized by the recognizer 130 in a predetermined situationare included, and, in a predetermined situation, in a case in which atleast a part of the road structure used for predicting the running locusof another vehicle in the future cannot be recognized by the recognizer130, the running locus predictor 140 predicts a running locus of anothervehicle in the future on the basis of the running locus of anothervehicle in the past acquired on the basis of a result of recognition inthe past acquired by the recognizer 130, whereby the running locus ofanother vehicle in the future can be predicted more continuously.

[Hardware Configuration]

FIG. 14 is a diagram illustrating one example of the hardwareconfiguration of an automatic driving control device 100 according to anembodiment. As illustrated in the drawing, the automatic driving controldevice 100 has a configuration in which a communication controller100-1, a CPU 100-2, a random access memory (RAM) 100-3 used as a workingmemory, a read only memory (ROM) 100-4 storing a boot program and thelike, a storage device 100-5 such as a flash memory or a hard disk drive(HDD), a drive device 100-6, and the like are interconnected through aninternal bus or a dedicated communication line. The communicationcontroller 100-1 communicates with constituent elements other than theautomatic driving control device 100. A program 100-5 a executed by theCPU 100-2 is stored in the storage device 100-5. This program isexpanded into the RAM 100-3 by a direct memory access (DMA) controller(not illustrated in the drawing) or the like and is executed by the CPU100-2. In this way, some or all of the recognizer 130, the running locuspredictor 140, and the action plan generator 150 are realized.

The embodiment described above can be presented as below.

A prediction device including a storage device storing a program and ahardware processor and configured such that the hardware processor, byexecuting the program stored in the storage device, recognizes a roadstructure and another vehicle in the vicinity of a subject vehicle,predicts a running locus of the other vehicle recognized by therecognizer in the future on the basis of the road structure recognizedby the recognizer in a predetermined situation and, in the predeterminedsituation, in a case in which at least a part of the road structure usedfor predicting the running locus of the other vehicle in the future isnot recognizable for the recognizer, predicts the running locus of theother vehicle in the future on the basis of a running locus of the othervehicle in the past acquired on the basis of a result of recognition inthe past that is acquired by the recognizer.

While preferred embodiments of the invention have been described andillustrated above, it should be understood that these are exemplary ofthe invention and are not to be considered as limiting. Additions,omissions, substitutions, and other modifications can be made withoutdeparting from the spirit or scope of the present invention.Accordingly, the invention is not to be considered as being limited bythe foregoing description, and is only limited by the scope of theappended claims.

What is claimed is:
 1. A vehicle control device comprising a processorthat executes instructions to: recognize a road structure and anothervehicle in the vicinity of a subject vehicle using at least one of acamera, a radar device, and a finder, wherein the road structure isdetermined by identifying a road partition line, a curbstone, a medianstrip, and a guardrail; predict a running locus of the another vehiclein the future based on a point of start turning recognized based on theroad structure, at least one point of finish turning predicted based onthe road structure and a position of the another vehicle in apredetermined situation; and perform automatic driving control bygenerating a target locus along which the subject vehicle runs in thefuture such that the target locus does not interfere with the predictedrunning locus of the another vehicle, wherein, in the predeterminedsituation, in a case in which at least a part of the road structure usedfor predicting the running locus of the another vehicle in the future isnot recognized, the processor further executes instructions to predictthe running locus of the another vehicle in the future based on arunning locus of the another vehicle in the past acquired based on aresult of recognition in the past, wherein the predetermined situationis a situation in which the another vehicle changes a course at anintersection, wherein the processor further executes instructions topredict the running locus of the another vehicle in the future based ona position of an entrance of the intersection and at least one exit ofthe intersection through which the another vehicle passes, which areacquired from the recognized road structure, and predict the runninglocus of the another vehicle in the future based on the running locus ofthe another vehicle in the past in a case in which one or both of thepositions of the entrance and the at least one exit are blocked,wherein, the point of start turning includes the entrance of theintersection and the at least one point of finish turning includes theat least one exit of the intersection, and wherein the processor furtherexecutes instructions to predict the running locus of the anothervehicle in the future by correcting a provisional running locuspredicted from the running locus of the another vehicle in the pastbased on the position of the at least one exit in a case in which theposition of the at least one exit is recognized, and the position of theentrance is not recognized.
 2. The vehicle control device according toclaim 1, wherein the predetermined situation is a situation in which theanother vehicle crosses a road and advances out of the road in a case inwhich the another vehicle is an oncoming vehicle.
 3. A vehicle controldevice comprising a processor that executes instructions to: recognize aroad structure and another vehicle in the vicinity of a subject vehicleusing at least one of a camera, a radar device, and a finder, whereinthe road structure is determined by identifying a road partition line, acurbstone, a median strip, and a guardrail; predict a running locus ofthe another vehicle in the future based on a point of start turningrecognized based on the road structure, at least one point of finishturning predicted based on the road structure and a position of theanother vehicle in a predetermined situation; and perform automaticdriving control by generating a target locus along which the subjectvehicle runs in the future such that the target locus does not interferewith the predicted running locus of the another vehicle, wherein, in thepredetermined situation, in a case in which at least a part of the roadstructure used for predicting the running locus of the another vehiclein the future is not recognized, the processor further executesinstructions to predict the running locus of the another vehicle in thefuture based on a running locus of the another vehicle in the pastacquired based on a result of recognition in the past, wherein thepredetermined situation is a situation in which the another vehiclechanges a course at an intersection, wherein the processor furtherexecutes instructions to predict the running locus of the anothervehicle in the future based on a position of an entrance of theintersection and at least one exit of the intersection through which theanother vehicle passes, which are acquired from the recognized roadstructure, and predict the running locus of the another vehicle in thefuture based on the running locus of the another vehicle in the past ina case in which one or both of the positions of the entrance and the atleast one exit are blocked, wherein, the point of start turning includesthe entrance of the intersection and the at least one point of finishturning includes the at least one exit of the intersection, and wherein,in a case in which the position of the exit is not recognized, theprocessor further executes instructions to estimate the position of theat least one exit based on a running locus of a preceding vehicle in thepast running in front of the another vehicle that is a target forpredicting the running locus in the future and predict the running locusof the another vehicle in the future.
 4. A vehicle control devicecomprising a processor that executes instructions to: recognize a roadstructure and another vehicle in the vicinity of a subject vehicle usingat least one of a camera, a radar device, and a finder, wherein the roadstructure is determined by identifying a road partition line, acurbstone, a median strip, and a guardrail; predict a running locus ofthe another vehicle in the future based on a point of start turningrecognized based on the road structure, at least one point of finishturning predicted based on the road structure and a position of theanother vehicle in a predetermined situation; and perform automaticdriving control by generating a target locus along which the subjectvehicle runs in the future such that the target locus does not interferewith the predicted running locus of the another vehicle, wherein, in thepredetermined situation, in a case in which at least a part of the roadstructure used for predicting the running locus of the another vehiclein the future is not recognized, the processor further executesinstructions to predict the running locus of the another vehicle in thefuture based on a running locus of the another vehicle in the pastacquired based on a result of recognition in the past, wherein thepredetermined situation is a situation in which the another vehiclechanges a course at an intersection, wherein the processor furtherexecutes instructions to predict the running locus of the anothervehicle in the future based on a position of an entrance of theintersection and at least one exit of the intersection through which theanother vehicle passes, which are acquired from the recognized roadstructure, and predict the running locus of the another vehicle in thefuture based on the running locus of the another vehicle in the past ina case in which one or both of the positions of the entrance and the atleast one exit are blocked, wherein, the point of start turning includesthe entrance of the intersection and the at least one point of finishturning includes the at least one exit of the intersection, and wherein,in a case in which the position of the exit is not recognized, theprocessor further executes instructions to estimate the position of theat least one exit based on the road structure on a side facing the atleast one exit at the intersection and predict the running locus of theanother vehicle in the future based on the estimated position of the atleast one exit.
 5. A vehicle control device comprising a processor thatexecutes instructions to: recognize a road structure and another vehiclein the vicinity of a subject vehicle using at least one of a camera, aradar device, and a finder, wherein the road structure is determined byidentifying a road partition line, a curbstone, a median strip, and aguardrail; predict a running locus of the another vehicle in the futurebased on a point of start turning recognized based on the roadstructure, at least one point of finish turning predicted based on theroad structure and a position of the another vehicle in a predeterminedsituation; and perform automatic driving control by generating a targetlocus along which the subject vehicle runs in the future such that thetarget locus does not interfere with the predicted running locus of theanother vehicle, wherein, in the predetermined situation, in a case inwhich at least a part of the road structure used for predicting therunning locus of the another vehicle in the future is not recognized,the processor further executes instructions to predict the running locusof the another vehicle in the future based on a running locus of theanother vehicle in the past acquired based on a result of recognition inthe past, wherein the predetermined situation is a situation in whichthe another vehicle changes a course at an intersection, wherein theprocessor further executes instructions to predict the running locus ofthe another vehicle in the future based on a position of an entrance ofthe intersection and at least one exit of the intersection through whichthe another vehicle passes, which are acquired from the recognized roadstructure, and predict the running locus of the another vehicle in thefuture based on the running locus of the another vehicle in the past ina case in which one or both of the positions of the entrance and the atleast one exit are blocked, wherein, the point of start turning includesthe entrance of the intersection and the at least one point of finishturning includes the at least one exit of the intersection, and wherein,in a case in which the position of the at least one exit is notrecognized, the processor further executes instructions to estimate theposition of the at least one exit based on a road width on a side facingthe at least one exit at the intersection and predict the running locusof the another vehicle in the future based on the estimated position ofthe at least one exit.
 6. A vehicle control device comprising aprocessor that executes instructions to: recognize a road structure andanother vehicle in the vicinity of a subject vehicle using at least oneof a camera, a radar device, and a finder, wherein the road structure isdetermined by identifying a road partition line, a curbstone, a medianstrip, and a guardrail; predict a running locus of the another vehiclein the future based on a point of start turning recognized based on theroad structure, at least one point of finish turning predicted based onthe road structure and a position of the another vehicle in apredetermined situation; and perform automatic driving control bygenerating a target locus along which the subject vehicle runs in thefuture such that the target locus does not interfere with the predictedrunning locus of the another vehicle, wherein, in the predeterminedsituation, in a case in which at least a part of the road structure usedfor predicting the running locus of the another vehicle in the future isnot recognized, the processor further executes instructions to predictthe running locus of the another vehicle in the future based on arunning locus of the another vehicle in the past acquired based on aresult of recognition in the past, wherein the predetermined situationis a situation in which the another vehicle changes a course at anintersection, wherein the processor further executes instructions topredict the running locus of the another vehicle in the future based ona position of an entrance of the intersection and at least one exit ofthe intersection through which the another vehicle passes, which areacquired from the recognized road structure, and predict the runninglocus of the another vehicle in the future based on the running locus ofthe another vehicle in the past in a case in which one or both of thepositions of the entrance and the at least one exit are blocked,wherein, the point of start turning includes the entrance of theintersection and the at least one point of finish turning includes theat least one exit of the intersection, and wherein, in a case in whichthe position of the at least one exit is not recognized, the processorfurther executes instructions to estimate the position of the at leastone exit based on a position of a structure part disposed near the atleast one exit and predict the running locus of the another vehicle inthe future based on the estimated position of the at least one exit.